ADA102105 


leMit  Ni.FU-CT-tl  fiO 


[EVEli^  (j 

DISCRETE  ADDRESS  DEACON  SYSTEM  (DADS)  ^ 
SOFTWARE  SYSTEM 

RELIADILITY  MODELING  AND  PREDICTION 

EVALUATION  ASSOCIATES,  INC. 

GSB  BLDG.,  I  BELMONT  AVE. 

BALA  GYNWYB,  PA.  19004 

Contract  No.  DTFAD3-B0-C-0002B 


FINAL  REPORT 
lUNE  I9BI 


I 


Dnrufti.T  :  I  .i(  • 

Ihf-  Ndl'iv  !.•  I  ■ 

Si.'  '  -r  '■  .1 


'  ■  ■  I J  S  i  Mliiii  ; hi i  )uqh 

I  il  «i'  S*''\  i,  6, 

V  Ii; '  ,,  1  hi 


Prepared  tor 

U.  S.  DEPARTMENT  OF  TRANSPORTATION 

FEDERAL  AVIATION  ADMINISTRATION 
TECHNICAL  CENTER 

Atlantic  City  Airport  New  lersey  08405 


'v>  w 


NOTICE 


This  document  is  disseminated  under  the  sponsorship  of 
the  Department  of  Transportation  in  the  interest  of 
information  exchange.  The  United  States  Government 
assumes  no  liability  for  the  contents  or  use  thereof. 


The  United  States  Government  does  not  endorse  products 
or  manufacturers.  Trade  or  manufacturer's  names  appear 
herein  solely  because  they  are  considered  essential  to 
the  object  of  this  report. 


2.  Goverrtmenf  Accession  No. 


Tachnicol  kaport  Oocumantafion  Page 


ATyA/o'^  /aj>' 


DISCRETE  ADDRESS  ^EACON^YSTEM  (DABS)  SOFTWARE  SYSTEM. 
^  BELIABILITY^ODELING  AND  ^REDICTION  • 


7  Ayrhof 


5.  Obf€  ^ 

f  Juatf  mi  ■' 

~6.  "^ertormTng  Orgoni  xotion  Cod®  , 

. _ i7'  -A 

Performing  Orgom  i^tion'^^vpor't  No. 


3.  Recipient's  Cotoiog  No. 

■  /r 


9.  Pvrformtng  Organization  Noma  ond  Address 

Evaluation  Associates,  Inc. 
GSB  Bldg.,  1  Belmont  Ave . 
Bala  Cynwyd,  Pa.  19004 


12  Sponsoring  Agency  Name  and  Address 

0  .  S .  Department  of  Transport  at  ion 
Federal  Aviation  Administration 
Technical  Center 

Atlantic  City  Airport,  New  Jersey  08405 

^5  Supplementory  Notes 


FAA-CT-81-60 


10.  Work  Unit  No  (TRAIS) 


iMr-eawtSFi'or'  rsrsrrrrro— 

DTFA0-3-8,0-C-OOO28 


.1 


13.  Type  of  Repart*>ohtt^etn>d  -Ceeefd.,...  ^ 

.9  .  99 

F inal  ^ 

January  1979  — June  / 


Spiansenng  Agewcy 


16  Abstroct 


^Thi.s  report  contains  the  results  of  a  pilot  study  accomplished  to  demonstrate  the 
abilitv  to  determine  the  magnitude  of  software  reliability  encountered  in  large-scale 
f  computer-based  equipment.  The  engineering  model  of  the  Discrete  Address  Beacon 
Svstem  (DABS)  currently  undergoing  development  was  used  as  the  subject.  Based  on 
software  failure  and  test  time  data,  a  software  reliability  model  was  developed  for 
the  engineering  model  of  DABS  and  used  to  measure  software  reliability  and  its  growth 
during  the  debugging  process.  The  software  reliability  model  was  merged  with  the 
hardware  reliability  model  into  a  DABS  system  model  suitable  for  prediction.  The 
i  Mean  Time  Between  Failures  (MTBF)  determined  by  this  study  applies  only  to  an  earlv 
version  of  the  software  associated  with  the  engineering  model  of  the  DABS.  The 
report  also  includes  recommendations  for  the  specification  of  software  reliability 
and  the  modification  to  the  failure  reporting  system, 
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1.  EXECUTIVE  SUMMARY. 


1.1  OBJECTIVE. 

The  Federal  Aviation  Administration  (FAA)  uses  mathematical  models  to 
measure  and  predict  the  reliability  of  hardware,  FAA  engineering 
specifications  for  systems  under  development  contain  reliability 
requirements,  usually  in  terms  of  mean-time-between-failures  (MTBF) , 
for  the  hardware  portions  of  these  systems.  However,  recently  pro¬ 
cured  systems  contain  computers  with  copious  amounts  of  software. 

It  has  been  the  experience  of  the  data  processing  community  that 
failures  of  such  systems  are  not  confined  to  hardware.  Software  which 
has  been  debugged  and  in  use  for  many  years  has  been  known  to  cause 
system  failure.  Consequently  the  FAA  initiated  this  pilot  study  to 
determine  the  magnitude  of  the  software  reliability  problem  in  one 
system  currently  in  development.  Study  objectives  were  the  following: 

-  Develop  a  software  reliability  model  for  the  Discrete  Address 
Beacon  System  (DABS)  and  make  a  software  reliability  prediction, 

-  Review  and  critique  the  available  hardware  reliability  model 
and  the  hardware  reliability  prediction  for  DABS. 

-  Integrate/evolve  the  software  and  hardware  reliability  models 
into  a  DABS  system  model  and  make  a  system  reliability  prediction. 

-  Compare  the  predicted  systems  reliability  value  versus  the 
specified  value.  Make  applicable  recommendations  for  reliability 
improvement  of  the  system. 

-  Recommend  a  software  reliability  failure  reporting  system  for 
the  DABS. 


1.2  _  BAG K G  R£Uf^ . 

The  objectives  cited  above  were  accomplished  by  grouping  related 
objectives  and  tasks  accordinj  to  importance  as  defined  by  Mr.  G. 
Apostolakls,  head  of  the  Reliability  Engineering  Section  at  the  FAA 
Technical  Center.  As  stated  by  Mr.  Apostolakls,  the  primary  concern 
of  the  FAA  is  the  study  of  DABS  software  reliability — how  it  could  be 
measured,  modeled,  predicted  and  how  it  could  be  incorporated  with  the 
hardware  into  an  integrated  software/hardware  reliability  model  for 
DABS.  The  results  of  this  study  are  contained  in  the  body  of  this 
report  ,  Sections  2  through  5.  Of  secondary  concern  are  1)  the  review 
and  critique  of  the  DABS  hardware  reliability  model  and  prediction, 
and  2)  a  recommended  software  reliability  failure  reporting  system  for 


the  DABS.  A  brief  critique  of  the  DABS  hardware  reliability  model  is 
contained  in  Appendix  A  to  this  report.  Because  the  FAA  already  has 
a  failure  reporting  system  for  DABS  software,  a  review  of  the  proce¬ 
dures  and  forms  was  made.  Recommendations  for  improvement  are  con¬ 
tained  in  Appendix  B  to  this  report. 

The  DABS  software  reliability  was  modeled  using  test  time  and  failure 
data  obtained  from  the  testing  of  the  sensors  at  three  test  sites--FAA 
Technical  Center,  Elwood  and  Clemcnton,  N.,1.  Based  on  tests  conductt-d 
between  February  1979  and  June  1980,  reliability  measurements  were  made 
lur  nine  software  modules  which  comprise  the  DABS  mission  software. 
Maintenance  and  off-line  software  were  not  modeled.  Also  not  modeled 
was  the  Automatic  Traffic  Advisory  and  Resolution  Service  (ATARS) 
module  because  of  its  interim  status. 

R(‘l  lability  prediction  niodels  for  software  modules  were  derived  and 
then  verified  by  matching  predictions  of  error  rate  with  software 
tv’st  ilata  collected  during  July,  August  and  September  of  1980.  Meas- 
ufuii^ents  of  software  reliability  obtained  from  the  models  were  com¬ 
bined  with  hardware  reliability  predictions  (prepared  by  FAA)  to 
obtain  nn  integrated  DABS  reliability  prediction  model. 

Uijiately ,  t'.ere  is  no  concenrsus  in  the  literature  pertaining  to 
M.(.'  .h.- f  i  n  i  t  ions  of  corcionly  used  terms  such  as  bugs,  errors,  faults 
and  failures.  A  few  definitions  are  presented  here  to  provide  tiie 
loader  some  insiglit  into  those  terms  and  concepts  of  software  reli- 
a  'lit'.'. 


-  Software  bugs,  errors  and  faults  will  be  considered  to  be 
c.yrion ymous  .  They  denote  latent  defects  present  in  software  due  to 
coding  errors,  misunderstanding  of  the  required  logic  on  the  ]  art  of 
the  t  rogrtcnmer ,  incorrect  algorithms  or  otlier  prograinming  Lrrurs. 

-  A  software  failure  occurs  when  certain  comloinati  ons  of  iiq  ut 
[.iiameters,  input  commands,  input  options  or  input  data  exeroi.,,L  t:,e 

I  ii  f  ,.-c  L  i  VL-  j  art  of  ttic  program.  Under  a  large  variety  of  ci  i  cu:;l  t  aiui  e, 
one  may  consider  these  inj')uts  to  be  random  sets  from  all  possible 
LiijUts.  TiiC'se  random  sots  of  inputs,  in  turn,  cause  random  failniLS 
in  t’K;  corresponding  outputs.  The  random  output  failures  may  be  ana - 
h  .'.('d  :,t  it  i  St  i  cal  ly  and  thus  constitute  the  liasis  for  the  concei>t  of 

If  liability  as  applied  to  software  failures. 

-  Software  reliability  is  the  f>robability  that  a  given  soft-ware 
I  logiain  will  operate  without  failure  for  a  spiocified  time  in  a 

:  ,r  I  c-  i  f  i  fd  <jnvi  ronment . 
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1.3  SIGNIFICANT  FINDINGS. 


1.  The  DABS  has  a  measured  overall  MTBF  of  252  hours;  575  hours  MTBF 
for  the  hardware  and  448  hours  for  the  software.  Only  critical  fail¬ 
ures,  those  which  dramatically  reduce  system  capability,  were  counted 
in  computing  the  MTBFs  of  hardware  and  software. 

This  achieved  MTBF  is  far  s)iort  of  the  ICiOO-hour  MTBF  specified  in 
the  engineering  reciuirements .  It  is  recognized  that  the  required 
MTBF  of  lOOO  iiours  was  intended  for  application  only  to  liardware, 
but  even  if  the  software  is  ignored  the  systi:m  does  not  now  meet  its 
reliability  requirement.  If  all  ciiargcable  software  failures  were 
included  in  tiie  calculation,  softwaz-e  MTBF  would  decrease  to  81  )iours 
and  the  system  MTBF  would  decrease  to  71  hours.  These  measurements 
are  based  on  a  total  of  5386  software  test  hours  during  which  354 
errors  wore  observed  and  evaluated. 

It  siiould  be  recognized  ttiat  DABS  is  undergoing  development  testing 
and  that  its  reliability  is  exi-ected  to  increase  as  improvements  are 
incoiq  orated ,  Also,  the  transition  from  a  test  or  debugging  scenario 
to  an  operational  scenario  siiould  noticeably  improve  the  measured  MTBF 
of  the  software.  Much  of  the  software  testing  at  the  Technical  Center 
was  geared  toward  pushing  the  system  to  its  sp.ecified  operational 
limits  ( o .q.,capac ity  testing,  multiple  correlations,  crossing  tracks). 
Tine  system  was  tested  using  a  multitude  of  inpiut  envi  ronments  and 
many  of  the  reported  errors  were  discovered  as  a  result  of  testing 
using  injiut  environments  w'nich  would  not  ordinarily  be  encountered 
in  an  oj'crational  scenario. 

2.  A  critical  software  failure  will  frequently  have  a  far  yi'oater 
effect  on  system  operation  than  a  computer  !iardwarc  failure  because 
critical  software  failures  cause  a  significant  or  complete  loss  of 
system  capability;  that  is,  they  defeat  Uie  iiardware  redundancy  built 
into  the  system.  In  the  event  of  computer  failure  the  system  can 
recover  by  using  a  spare  comp.uter;  however,  critica]  software  ‘ai  lures 
result  in  either  complete  system  failure  or  reduced  pierformanco  which 
docs  not  meet  specification.  From  a  reliability  point  of  view, 
partial  system  operation  is  considered  to  be  a  failed  condition 
because  no  reliability  reejuirements  are  sj-ecified  for  alternate 
(degraded)  modes  of  operation. 

It  is  recommended  that  the  FAA  investigate  the  design  of  fault- 
tolerant  software  for  DABS.  The  software  could  be  designed  to  :',cnse 
critical  software  failures  (watchdog  logic  or  audits)  which  would 
recover  the  system  in  much  the  same  fashion  as  a  computer  failure 
by  causing  an  automatic  re-initialization  of  the  system. 
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3.  The  Duane  reliability  growth  model  which  has  been  used  extensively 
to  model  the  growth  of  hardware  reliability  and  more  recently  to  model 
the  growth  of  software  reliability  as  well,  fits  the  known  history  of 
DABS  software.  Of  the  nine  software  modules  in  DABS,  the  Duane  model 
accurately  predicts  reliability  and  rate  of  reliability  growth  of 
five  modules.  Tlie  modules  and  their  rate  of  reliability  growth  models 
are : 


- .  503 

Communication:  Aj;  =  .174  T 

Measured  MTBF  at  end  of  study:  976  hours 

-.419 

Performance  Monitoring:  A^  =  .1403  T 
Measured  MTBF  at  end  of  study:  494  hours 

-.645 

Message  Routing  &  Data  kink:  =  .3467  T 

Measured  MTBF  at  end  of  study:  2400  hours 

System  Software:  Aj;  =  5.689  T 

Measured  MTBF  at  end  of  study:  2588  hours 

C  -11  1  t  AA-7  'r~'  3071 

Surveillance:  Ar  =  .1067  f 

Mi-asured  MTBF  at  end  of  study:  207  hours 

viiore  Ay;  =  cuniulative  error  rate  (number  of  chargeable  errors/total 
test  hours)  and  T  =  cumulative  test  hours.  Of  the  remaining  modules, 
the  data  were  either  too  sparse  to  evaluate  the  parameters  of  the 
iM'dt'l  or  too  erratic  to  determine  whether  niodule  reliability  is 
L::.:  i  iiv ing . 

'Hie  parameters  appearing  as  exponents  of  T  indicate  the  rate  of  MTBF 
growth  (MTBF  =  1 /error  rate),  whicli  is  usually  a  measure  of  manage- 
liient  jiressure  to  find  and  correct  errors.  The  rates  shown  are  all 
within  the  range  typically  encountered  but  they  vary  more  than  usual. 
Inis  suggests  that  testing,  debugging  and  integration  efforts  have 
I'lot  iieen  applied  uniformly  in  the  DABS  program.  Some  modules  have 
rotnived  much  more  attention  than  others. 

4.  Rased  on  hardware  reliability  measurements  reported  in  Report  No. 
F.\.\-RD- 80- 36 ,  "Discrete  Address  Beacon  System  (DABS)  Baseline  Test  and 
i'val  uat  ion, "  April  1980,  DABS  hardware  MTBF  growth  rate,  albeit  using 
a  iiiuill  sample,  was  calculated  to  be  a  =  .36.  Projections  of  hard¬ 
ware  MfBF  inprovement  icsing  ci  =  .  36  and  software  MTBF  improvement  using 
i  =  .52  show  that  the  DABS  sof tware /hardware  system  will  achieve  its 
inOO-hour  IffBF  requirement  after  49  additional  months  of  testing. 

At  that  time  hardware  MTBF  will  be  approximately  1650  hours  and  soft¬ 
ware  MIBF  will  be  approximately  2500  hours  if  the  growth  rate 
,  t  1  rules  . 
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The  models  predict  that  if  no  changes  are  made  in  the  present  reli¬ 
ability  improvement  efforts,  software  errors  will  still  constitute  10 
piercent  of  total  system  errors  (based  on  1000-hour  hardware  MTBF)  after 
50  X  10^  software  test  hours  (test  time  needed  to  achieve  software 
MTBF  =  10,000  hours). 

The  following  actions  are  recommended  to  speed  up  the  reliability 
improvement  of  the  DABS  system: 

-  Increase  the  intensity  of  the  software  test  program  to  con¬ 
duct  well  planned  non-random  testing  such  as  the  identification  and 
evaluation  of  degraded  as  well  as  complex  inputs  to  software  modules. 

-  Automatically  identify/isolate  access  of  the  code  with  low 
input/output  traffic;  check  all  jump  statements. 

-  Conduct  failure  modes,  effects  and  criticality  analyses  of 
hardware  elements  which  contribute  most  to  unreliability--antenna , 
transmitter,  receiver  and  processor.  Tlicse  elements  which  have  no 
redundancy  in  the  single  channel  sensor  could  be  improved  through  the 
idenfication  of  failure  modes  and  their  elimination  through  correc¬ 
tive  action. 

-  The  reliability  engineering  department  of  the  FAA  Teclinical 
Center  should  continue  to  monitor  the  progress  of  the  software  test, 
j-articipate  in  configuration  management  and  include  estimates  of 
software  reliability  in  DABS  predictions. 

5.  Analysis  of  the  test  data  for  the  purpose  of  measuring  reliability 
and  its  growth  showed  that  the  error  rates  of  most  software  modules 
changed  appreciably  during  the  test.  Several  causes  are  postiilated: 

a)  As  noted  in  Figure  4,  the  nearly  constant  error  rate  of  the 
communication  module  for  900  test  hours  is  followed  by  a  rapidly 
declining  error  rate.  This  pattern  is  characteristic  of  an  early 
test  period  in  which  the  software  package  was  not  tested  with  the 
intensity  needed  to  identify  and  correct  errors.  Subsequent  testing 
then  resulted  in  the  identification  and  elimination  of  more  errors  at 
a  significantly  higher  rate. 

b)  Software  test  personnel  are  sometimes  reluctant  to  document 
errors  as  they  are  observed  because  they  believe  that  continuation 
of  the  test  and  analyses  of  results  are  more  important  tasks.  Con¬ 
sequently,  failures  may  be  documented  en  masse  several  weeks  or 
months  after  they  occur,  usually  at  the  completion  of  the  test. 

Such  perturbations  to  the  model  may  require  several  additional  data 
points  to  effect  smoothing. 
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c)  Neither  the  Duane  nor  any  other  model  which  assumes  a  con¬ 
tinuously  decreasing  error  rate  with  time  can  predict  significantly 
large  perturbations  due  to  mass  introductions  of  software  modifica¬ 
tions  at  "release"  points.  These  can  either  increase  or  decrease  an 
error  rate.  Abrupt  termination  of  a  debugging  process  will  also  sig¬ 
nificantly  reduce  the  observed  error  rate. 

6.  Tlie  f'AA  should  endeavor  to  include  software  as  well  as  hardware 
elements  in  future  reliability  models  for  DABS  and  other  computer 
aided  systems.  Reliability  requirements  of  future  systems,  which 
are  often  set  by  systems  requirements  analyses,  should  also  include 
the  reliability  of  the  software.  Measured  reliability  of  the  system 
will  tlien  be  realistic  since  it  will  apply  to  software  and  hardware. 


2. 


DESCRIPTION  OF  DABS  COMPUTER  SOFTOARE,  TESTING  AND  DATA  BASE. 


2.1  DESCRIPTION  OF  DABS  COMPUTER  SOFTWARE  AND  ITS  TESTING. 


As  described  by  Dr.  C.  M.  Applewhite  in  "Disbributed  Computer  Architec¬ 
ture  For  The  Discrete  Address  Beacon  System,"  the  purpose  of  DABS  is  to 
provide  highly  reliable  tracking  and  collision  avoidance  support  for 
DABS-equipped  aircraft.  Control  of  DABS  is  provided  by  software 
operating  in  a  ground  based  distributed  computer  network  interfaced 
to  a  beacon  radar.  Each  DABS  aircraft  is  assigned  a  unique  identifica¬ 
tion  (discrete  address)  associated  with  its  DABS  transponder.  Recog¬ 
nition  of  a  beacon  interrogation  is  keyed  to  the  discrete  address  of 
each  particular  aircraft  such  that  a  unique  data  link  with  miminum 
interference  can  be  established  between  the  computer  network  and  each 
aircraft.  The  software  subsystem  maintains  a  track  update  on  each 
aircraft,  predicts  potential  conflict  situations  and  controls  the 
scheduling  of  the  beacon  radar.  Data  to  support  aircraft  tracking  is 
gathered  via  uplink  interrogations  and  downlink  responses  of  aircraft 
positional  data.  Traffic  data  and  maneuver  advisories  are  jirovided 
to  the  pilots  via  the  uplink  in  the  event  the  computer  subsystem 
predicts  a  potential  conflict  situation.  Telephone  line  data  links 
between  sensors  facilitate  coordination  among  adjacent  seiisors  with 
overlapping  airspace  responsibilities. 

DABS  surveillance  capability  is  designed  to  be  completely  compatible 
with  the  present  Air  Traffic  Control  Radar  Beacon  System  (ATCKBS)  and 
thus  can  be  introduced  gradually  and  economically  wit'nout  major  oj-er- 
ational  or  procedural  change.  Since  DABS  uses  monopulse  direction 
finding,  the  system  also  provides  improved  surveillance  cov'erage  for 
ATCRBS  equipped  aircraft  at  a  reduced  interrogation  rate. 

In  addition  to  the  requirements  given  above,  the  software  system  is 
required  to  respond  to  computer  hardware  failures  by  reconf igur ing 
the  system  and  maintaining  system  integrity,  to  monitor  system  status 
indicators,  to  send  status  messages  to  ATC  maintenance  facilities  and 
to  collect  performance  data  for  the  sensor.  A  functional  block  dia- 
cjram  which  highlights  some  of  the  D/iBS  features  is  shown  in  Figure  1. 
The  architecturally  distributed,  molecular  software  is  shown  in  Fig¬ 
ure  2 . 

No  special  tests  were  run  expressly  to  provide  data,  solely  for  reli¬ 
ability  analysis.  Consequently,  the  running  time  and  errors  generated 
during  debugging,  checkout  and  operation  of  the  DABS  sensors  at  FAA 
Technical  Center,  Elwood  and  Clementon,  N.J. ,  were  used  to  formulate 
the  software  reliability  model  and  measure  achieved  reliability  and 
growth  rate.  The  DABS  software  was  tested  formally  and  informally. 

At  the  FAA  Technical  Center,  initial  testing  was  conducted  by  running 


Figure  2,  DABS  Software  Independent  Task/Processor  Organizati 


the  system.  Maximum  specified  values  of  targets  and  fruit  rates 
(replies  to  interrogations  from  other  sensors)  were  simulatt;d  to  test 
DABS  software  and  hardware.  The  test  program  uncovered  coding  errors, 
timing  and  interrupt  faults. 

2.2  DABS  SOJ-'TOAAE  PAJA  PAPA- 

The  software  test  time  base  is  shown  in  Figure  3.  Tlie  figure  contains 
monthly  estimates  of  software  test  time  for  three  facilities  where 
testing  occur red--FAA  Technical  Center,  Elwood  and  Cleinenton,  NJ. 
Beginning  in  October  1979  the  test  time  is  the  scheduled  software  test 
time  for  each  test  site.  Prior  to  October,  software  test  times  were 
based  on  estimates  obtained  from  Messrs.  M.  Holtz,  DABS  Program  Manag,er, 
and  J.  Simpfenderfer,  T.  I.  Technical  Representative.  The  figure  also 
shows  the  time  phasing  of  the  testing  of  the  various  DABS  software 
modules.  It  shows,  for  example,  that  channel  management,  surveillance 
and  data  extraction  were  considered  to  be  undergoing  test  from  the 
I'egiiining  of  the  test,  whereas  network  management  was  not  tested  until 
all  three  sensors  were  on-line  in  October  1979. 

An  additional  three  months  of  test  data  (1790  hours)  accrued  during 
July,  August, and  September  1980.  lliis  time  period  constituted  a 
small  controlled  sample  from  DABS  testing  to  be  used  to  verify  relia¬ 
bility  predictions  made  using  the  data  base  described  above.  This 
procedure  is  described  in  Section  3  of  this  report. 

Software  errors  discovered  during  the  test  programs  were  reported  on 
trouble  reports.  A  brief  description  of  the  reporting  system, 
including  a  sample  form,  is  given  in  Appendix  B  to  this  report. 

ITie  DABS  software  error  data  base  consists  of  354  trouble  rejiorts  (TR) 
which  document  program  stops,  errors,  enhancements  and  change  pro¬ 
posals.  Not  all  supply  adequate  information  for  reliability  analysis. 
Software  design  engineers  at  Texas  Instruments  reviewed  each  TR  and 
its  associated  follow-up  documentation  and  classified  the  TRs  with 
respect  to  severity.  Chargeable  errors  which  were  used  to  measure 
software  reliability  were  classified  as  critical  (1)  or  non-critical 
(2).  Tliose  not  chargeable  were  classified  other  (3)  or  no  count  (4). 
The  following  definitions  were  applied. 

Cliargeable  Errors: 

1.  Critical  -  An  error  in  the  software  which  causes  a  sig¬ 
nificant  or  total  loss  of  operational  system  capability. 

2.  Non-Critical  (Major)  -  An  error  in  the  software  which 
causes  an  erroneous  response  in  the  operational  system.  An  error  in 
this  classification  may  not  be  recognized  as  such  by  a  trained 
observer  due  to  the  self-repair  inherent  in  the  system. 
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Figure  3.  DABS  Software  Test  Baseline 


Non-Chargeable  Errors: 


3.  Other  (Minor)  -  An  error  which  has  no  measurable  effect 
on  the  operational  system  or  is  of  unknown  cause  at  this  time  (hard- 
ware/sof tware/cockplt) .  Errors  of  unknown  causes  would  be  charged 
against  the  DABS  system  rather  than  the  software. 

A.  No  Count  -  A  trouble  report  which  was  erroneously  attrib¬ 
uted  to  software  errors.  In  addition,  change  proposals,  enhancements, 
duplicate  trouble  reports  and  "cockpit  errors"  are  included. 

A  suitunary  of  chargeable  errors  is  presented  in  the  matrix  in  Table  1. 
Only  software  modules  identified  in  the  table  were  included  in  the 
reliability  analysis.  Other  software  modules  whicli  are  of f-1 ine analy¬ 
sis  tools  or  are  used  during  maintenance  or  pre-initialization  were 
not  modeled  because  they  are  not  part  of  the  mission  software.  The 
ATARS  module  which  will  eventually  constitute  a  large  portion  of  the 
DABS  software  subsystem  cannot  be  analyzed  now  because  it  is  being  re¬ 
written  and  is  not  sclteduled  for  extensive  testing  until  Spring  of  19S1. 
A  computer  listing  of  all  DABS  trouble  reports  is  contained  in  Appen¬ 
dix  C. 
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3.  APPROACH  TO  SOFTWARE  RELIABILITY  MODELING. 


3-J_  THEORETICAL  SOFTWARE  RELIABILITY  MODEL. 

Tlie  reliability  growth  model  Introduced  by  Duane  in  1964  and  more 
recently  expounded  by  Codier,  has  found  wide  acceptance  by  reliability 
engineers.  It  is  simple  to  use  and  it  is  applicable  to  both  continu¬ 
ous  and  discrete  data  cases.  Its  wide  applicability  to  diverse  hard¬ 
ware  test  programs  and  more  recently  to  software  test  data  prompted  its 
use  here. 

Using  data  from  several  different  types  of  hardware  test  programs  as  a 
basis,  Duane  plotted  cumulative  failure  rate  (Ay|)  vs.  total  operating 
time  (t)  and  observed  a  linear  relationship  between  log  (X^)  and  log 
(T)  for  each  equipment.  This  relationship  is  characterized  by  the 
model : 

=  KT  ^  where, 

=  cumulative  failure  rate 

K  =  a  model  parameter  to  be  estimated  (represents  X  at  T=l) 

T  =  total  operating  hours,  cycles  or  missions 
a  =  Growth  rate  to  be  estimated. 

Duane  presents  a  method  for  estimating  the  model  parameter  directly 
from  the  data  plotted  on  log-log  paper.  The  growth  parameter  can  be 
obtained  by  calculating  the  slope  of  the  line.  The  location  parameter 
is  also  obtained  directly  from  the  plot  as  the  value  for  Xv  at  T  =  1. 

K  can  be  interpreted  as  the  initial  or  zero-age  failure  rate.  For 
software,  it  is  a  function  of  program  complexity,  size,  its  imiturity 
relative  to  the  state-of-the-art  and  other  variables. 

The  curve  is  more  sensitive  to  the  exponent  a  than  to  K.  Tlie  exponent 
reflects  the  intensity  with  which  reliability  improvement  is  pursued; 
it  nearly  always  lies  between  .2  and  .5,  the  average  being  close  to  .3. 


In  addition  to  information  regarding  cumulative  failure  rates,  the 
predicted  failure  rate  at  any  point  in  time;  i.e.,  instantaneous 
failure  rate  Xj.,  can  also  be  estimated  from  the  following  equation 
where  F  =  total  failures  and  all  other  variables  have  been  previously 
defined. 


9T  ^  3T 


=  (l-a)KT“°‘  =  (1-«)X^ 
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Thus,  program  progress  can  be  modeled  using  cumulative  information  and 
can  be  continuously  monitored  using  the  current  information. 


3.2  DATA  ANAI.YSIS. 

The  operating  time  and  error  data  base  for  each  software  module  was 
analyzed  to  provide  inputs  into  the  Duane  model.  Using  chargeable 
errors  and  test  time  the  cumulative  error  rate  =  total  failures/ 
total  time  was  calculated  for  each  month  in  which  at  least  1  error 
was  reported.  The  data  were  plotted  in  accordance  with  the  Duane 
growth  curve  requirements  and  model  parameters  were  estimated  if 
growth  were  evident. 

The  modeling  process  generated  a  model  of  cumulative  error  rate, 

Xj-  =  KT"*^,  and  a  model  of  instantaneous  error  rate,  X^.  =  (l-a)Xj;  . 

The  reciprocals  of  error  rates  are  the  MTBFs ,  cumulative  and  instanta¬ 
neous  respectively.  In  addition,  MTBCF,  i.e.,  mean  time  between 
critical  (severity  class  1)  failures,  values  were  calculated  where 
applicable . 

For  the  modules  where  reliability  improvement  was  evident,  the  models 
were  used  as  predictive  tools  to  estimate  the  error  rate  during  future 
testing.  In  this  analysis  the  future  consisted  of  the  1790  test  hours 
during  July,  August  and  September  of  1980.  Results  of  the  predictions 
were  then  compared  to  actual  data.  The  analysis  of  the  COM  module 
(Section  4.1)  serves  as  a  detailed  example  of  the  process.  For  informa¬ 
tion  purposes,  Table  2  contains  a  listing  of  chargeable  errors  written 
during  the  prediction  test  interval. 
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Table  2,  Chargeable  Failures  Reported  During  the  Prediction  Interval 


Report 

Number 

Date  of 

Error 

Module 

Severity 

M0002 

7/10/80 

CM 

1 

N0066 

7/11/80 

MTD 

2 

S0317 

7/  16/8) 

PM 

2 

S0319 

7/16/80 

PM 

2 

S0320 

7/16/80 

PM 

1 

S0321 

7/18/80 

SYS 

1 

S0326 

7/20/80 

COMM 

2 

S0328 

7/23/80 

SURV 

2 

SO  330 

7/24/80 

CM 

O 

S0331 

7/24/80 

SYS 

2 

S0333 

7/24/80 

SURV 

1 

N0072 

8/11/80 

SURV 

2 

S033A 

8/  4/80 

COMM 

2 

S0335 

8/  4/80 

COMM 

2 

S0337 

8/  4/80 

COMM 

2 

S0338 

8/  4/80 

COMM 

2 

M0004 

9/29/80 

SURV 

2 

M0005 

9/29/80 

SURV 

2 

M0006 

9/29/80 

SURV 

2 

M0008 

9/29/80 

SURV 

2 

MOO  14 

9/29/80 

DEX 

2 

S0358 

9/15/80 

SYS 

2 
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4.  RELIABILITY  EVALUATION  OF  DABS  SOFTWARE  MODULES, 


4,1  RELIABILITY  EVALUATION  OF  THE  COMMUNICATION  MODULE . 

The  Communications  (COM)  Module  Includes  the  surveillance  and  CIDIN 
communications  programs  which  control  and  monitor  transfer  of  data 
between  a  sensor  and  external  facilities.  The  reliabilities  of  other 
software  in  the  intersite  communications  package  were  not  modeled 
because  tfie  software  is  associated  with  off-line  and  maintenance  pro¬ 
cessing  and  is  not  part  of  the  mission  software. 

The  COM  module  was  tested  for  4966  hours  during  which  11  chargeable 
errors  were  reported  (see  Table  3).  Only  4091  test  hours  along  with 
the  11  errors  were  used  to  construct  the  growth  model  because  the 
data  base  was  terminated  at  the  last  reported  failure  in  accordance 
with  the  rules  of  model  construction.  Results  of  the  model  genera¬ 
tion  and  reliability  predictions  follow. 

The  model  which  describes  cumulative  error  rate  is  Xj;  =  .174 
where  Xj;  =  cumulative  error  rate  and  T  =  cumulative  test  hours.  For 
example,  at  the  time  of  the  last  reported  error  (T  =  4091  hr.)  the 
model  predicts  Xj;  =  ,00265  error/hr,  or  377  hr.  MTBF.  Tlie  measured 
error  rate  at  T  =  4091  hr.  was  .0027  error/  hr.  or  372  hr.  MTBF.  UTien 
used  as  a  predictive  tool  to  extrapolate  beyond  the  time  limits  of 
the  data  base  to  T  =  6756  hours,  X^  =  ,174  (6756)"* 503  =  .00206  error/ 
hr.  or  485  hours  MTBF.  The  time  interval  between  4091  hours  and  6756 
hours  (2665  hours)  includes  the  last  875  hours  of  test  without  a 
reported  failure  and  1790  hours  of  test  during  the  prediction  interval. 
Because  the  model  predicted  a  cumulative  error  rate  of  .00206  error/hr. 
at  T  =  6756  hr.,  the  expected  number  of  cumulative  errors  was  calcu¬ 
lated  from:  F  =  Xj;  •  T  =  .00206  error/hr.  •  6756  hr.  =  135  errors. 
Because  11  errors  had  already  been  reported  within  4091  test  hours, 
the  remaining  2.9  errors  represent  a  prediction  to  be  compared  with 
the  observed  results.  Tn  fact,  five  chargeable  errors  were  reported 
against  the  COM  module,  a  number  which  is  well  within  the  limits  of 
statistical  variation.  Figure  4  contains  a  graph  of  the  model. 

The  model  which  describes  Instantaneous  error  rate  and  MTBF  is 
Xf.  =  .0865  T~'^®^.  It  represents  the  rate  at  which  errors  are  system¬ 
atically  being  identified  and  removed  from  the  COM  module  at  time  T 
hours.  For  example,  at  T  =  6756  hours,  X^  =  .0865  (6756)~‘^*^^  = 

.00102  error/hr.  or  976  hr.  MTBF.  The  value  of  X^  at  T  =  6756  has 
the  significance  that  if  the  test  correction  process  were  to  cease  at 
T  =  6756  hours,  the  error  rate  of  the  COM  module  would  no  longer 
decrease,  but  w’ould  become  constant  at  X  =  .00102  error/hr.  or  976  hr. 
MTBF. 
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Table  3.  Communication  Module  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

Cumulative 
Test  Hrs. 

(Tj) 

Cumulative 

Errors 

(Fj) 

Cumulat ive 
Error/Hr 

(X,) 

Average 

Time 

Between 

Er  rors 
(Mi  HFj-) 

1979  Feb 

Mar 

Apr 

May 

140 

1 

140 

1 

.0071 

140 

June 

140 

1 

280 

2 

.0071 

140 

July 

188 

1 

468 

3 

.0064 

156 

Aug 

188 

1 

656 

4 

.0061 

1  64 

Sept 

188 

1 

844 

5 

.0059 

1 

169 

Oct  1 

305 

0 

1149 

Nov 

390 

0 

1539 

1 

1979  Dec 

489 

0 

2028 

1980  Jan 

528 

1 

2556 

6 

.0023 

4  26 

Feb 

480 

3 

3036 

9 

.003 

337 

Mar 

431 

1 

3467 

10 

.0029 

347 

Apr 

624 

1 

4091 

11 

.0027 

372 

May 

418 

4509 

June 

457 

4966 

J  u  ly 

Aug 

Sept 

1790 

6756 

18 


00  1,000  13,000 

Cumulative  Test  Hours 


Figure  4.  Communication  Module  -  Reliability  Growth  Model 
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4.2  RELIABILITY  EVALUATION  OF  THE  PERFORMANCE  MONITORING  MODUl.E. 

The  Performance  Monitoring  (PM)  module,  a  portion  of  intrasite  conutiu- 
nications.is  responsible  for  gathering  and  analyzing  status  of  the 
sensor  and  the  transmission  of  status  messages  to  external  facilities. 

As  reported  in  Table  4,  the  PM  software  was  tested  for  4966  hours 
during  which  20  chargeable  errors  were  reported.  Of  the  chargeable 
errors,  five  were  classified  as  critical.  Figure  5,  which  contains 
the  reliability  growth  curve  for  the  PM  module  shows  that  the  module 
experienced  a  decreasing  error  rate  throughout  the  test  except  for 
minor  fluctuations.  The  cumulative  error  rate  model,  ~  .  140  3T~  • 
gives  ~  .00348  error/hr.  at  T  =  6756  hours.  This  translates  into 
3.5  expected  errors  during  the  time  of  the  test  interval  used  for  pre¬ 
diction  purposes.  Because  3  chargeable  errors  were  reported  during 
the  1790  test  hours  of  the  prediction  interval,  there  is  close  agree¬ 
ment  and  acceptance  of  the  model. 

-.419 

Tlie  instantaneous  error  rate  model,  =  .0815T  ’  ,  predicts  that 

A  =  .00203  error/hr.  at  T  =  6756  which  is  equivalent  to  494  hours  MTBF. 

'^.-.3  _KELJ.AB_IJ^irY  EVALUAllON  OF  MESSAGE  ROUTING  AND  DATA  LINK  PROCESSING 
m'nui.ES .~ 

Tlie  Message  Routing  (MR)  software  is  responsible  for  routing  incoming 
messages  to  the  appropriate  software  module.  Data  Link  (DL)  procL'Ssing 
manages  uplink  and  downlink  messages  to/from  participating  DABS 
i5::uiT  pod  aircraft.  MR  &  DL  software  were  tested  togetlier  and  form  a 
single  software  module  for  the  purpose  of ' reliability  analysis. 

Table  5  contains  the  time  and  error  data  used  to  generate  the  relia¬ 
bility  growth  curve  shcH'^n  in  Figure  6.  Using  the  cumulative  growth 
model,  Aj-  =  .3467  At  =  .00117  error/hr.  at  T  =  6756  hours, 

llie  model  predicts  the  occurrence  of  7.9  errors  throughout  the  test 
of  6756  hours.  Because  7  errors  were  reported  during  the  initial 
test  phases,  0.9  errors  were  predicted  to  occur  during  the  prediction 
test  interval.  Actually,  no  failures  were  reported  during  tiie  1790 
tiours  of  additional  test,  a  value  within  acceptable  statistical  vari- 
at ion . 

- .  645 

Tlie  instantaneous  error  rate  model,  A^^  =  .123T  ’  ,  predicts  A  = 

.000417  error/hour  or  2400  hours  MTBF  at  T  =  6756  hours. 
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Table  A.  Performance  Monitoring  Module  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

Cumulative 

Test  Hrs. 

(Tj) 

Cumul at ive 
Errors 

(Fj.) 

Cumul at ive 
Error/Hr 

1979  Feb 

- 

Mar 

Apr 

.  .. 

- 

May 

140 

3 

140 

3 

.0214 

June 

140 

0 

280 

July 

188 

2 

468 

5 

.0107 

Aug 

188 

0 

656 

Sept 

188 

1 

844 

6 

.0071 

Oc  t 

305 

5 

1149 

11 

.  0096 

Nov 

390 

0 

1539 

1979  Dec 

489 

1 

2028 

12 

.0059 

1980  Jan 

528 

1 

2556 

13 

.0051 

Fob 

480 

3036 

Mar 

431 

3467 

Apr 

624 

4 

4091 

17 

.004  2 

May 

418 

7 

4  509 

19 

.  0042 

June 

457 

1 

4966 

20 

.0040 

July 

-  , 

Aug 

1790 

Sept 

6756 

Average 
T  i  me 
Between 
Errors 
(Mi'BF^-.) 


47 


93 

141 


104 


169 


196 


2  38 
2  38 
230 


■ror/Hr) 


Table  5.  Message  Routing  and  Data  Link  Modules  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

1 

1 

Cumulative 
Test  Hrs. 

(Tj) 

1 

Cumulative 

Errors 

Cumulat Ive 
Error/Hr 

1979  Feb 

■  ■  '  . . 

- - -  -  . 

— 

Mar 

1 

Apr 

May 

140 

1 

140 

1 

.0071 

June 

140 

280 

1 

July 

188 

2 

468 

3 

.0064 

Aug 

188 

656 

Sept 

188 

1 

844 

4 

.0047 

Oct 

305 

..... 

1149 

... 

Mov 

390 

1539 

1979  Dec 

489 

2028 

1980  Jan 

528 

-  - -  . 

2556 

•  -  -  -  - 

Feb 

480 

3036 

Mar 

431 

2 

3467 

6 

.0017 

Apr 

624 

1 

4091 

7 

.0017 

May 

418 

4509 

June 

457 

4966 

July 

Aug 

1790 

Sept 

6756 

Average 

Time 

Between 

Errors 

(MTBFj,) 


141 


156 


213 


5  88 


588 


ve 


4.4  RELIABILITY  EVALUATION  OF  DATA  EXTRACTION  MODULE. 


Included  in  this  evaluation  are  data  from  the  portion  of  the  Data 
Extraction  (DEX)  module  which  is  associated  with  data  collection;  i.e., 
on-line  real  time  extraction  of  performance  data  from  the  DABS  data 
base  and  its  recording  on  magnetic  tape.  Playback,  quick-look  and 
extended  analysis  software  are  used  off-line  and  are  not  part  of 
the  mission  software. 

As  seen  in  Table  6  and  Figure  7,  the  DEX  module  is  very  reliable.  Only 
two  chargeable  errors  were  reported  in  5386  test  hours  for  an  MTBF  of 
2693  hours  or  an  error  rate  of  .000371  error/hr.  One  of  the  errors 
was  classified  as  critical.  Too  few  data  are  available  to  generate  a 
reliability  trend  curve.  It  can  be  seen  however,  that  the  error  rate 
is  decreasing  which  implies  that  the  Instantaneous  MTBF  is  greater 
than  2693  hours.  There  was  one  error  reported  against  DEX  software 
during  the  prediction  test  interval. 

i.-  5  RELIABILITY  EV.UUATION  OF  CHANNEL  MA.N'AGElfliNT  _>p PULE . 

Cliannel  Management  (CM)  regulates  all  activity  on  the  RF  channel, 
scheduling  tlie  aircraft  interrogations  and  corresponding  listening 
periods  to  ensure  that  conmiunications  and  surveillance  tasks  are 
accei-ipl  ished  for  each  aircraft. 

Tlie  CM  module  has  been  ctiaracteri  zed  by  T.  I.  software  designers  as 
the  most  cotiplex  of  the  DABS  software  modules  primarily  because  of  its 
Icigical  structure.  Its  measured  reliability  is  among  the  lowest. 

During  5386  hours  of  test,  14  ciiargeable  errors  were  reported  for  an 
error  rate  of  .0026  error/hr.  or  385  liours  M'J'BF.  Table  7  contains 
cumulative  time  and  error  data  for  CM.  Tlic  data  plot  in  Figure  8 
shows  that  no  trend  analysis  is  possible  because  of  the  abrupt  changes 
in  slope  of  tlie  curve.  In  fact,  during  the  test  period  between  2500 
hr.  and  5000  hr.  CM  error  rate  increased  from  .0016  error/hr.  to 
.0028  error/hr.  Subsequent  testing  indicates  a  reversal  of  the  trend 
because  the  error  rate  appears  to  be  decreasing  in  the  prediction 
interval  between  4929  hours  and  7176  hours.  Consequently,  for  predic¬ 
tion  purposes  the  cumulative  error  rate  X-  ~  total  errors/total  hours  = 
.0026  error/hr.  will  be  used. 


4.6  REIMABILm  EVALUATION  OF  NETOORK  MANAGEMENT  MODULE. 

Network  Management  (NM)  is  a  portion  of  intrasite  communications 
responsible  for  communication  of  surveillance  data  to  and  from  other 
sensors . 

Table  8  contains  the  data  irsed  in  the  reliability  analysis  of  the  NM 
software.  Based  on  a  total  of  4122  test  hours  and  22  chargeable  errors 


Table  6.  Data  Extraction  Module  -  Reliability  Data  Summary 


Monthly 

Monthly 

Cuinula 

Test  Hrs 

Errors 

Test  H 

Month 

(T) 

(F) 

(Tj.) 

1979  Feb 

lAO 

140 

Mar 

140 

280 

Apr 

140 

- - 
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May 

140 
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June 

140 

700 

July 

188 

1 

888 

Aug 

188 

1076 

Sept 

188 

1264 

Oct 

305 

_ 

1569 

Nov 

390 

1 

1959 

1979  Dec 

489 

2448 

1980  Jan 

528 

2976 

Feb 

480 

3456 

Mar 

431 

3887 

Apr 

624 

■  ‘  -  -  '  - 

4511 

May 

418 

4929 

June 

457 

5386 

July 

Aug 

1790 

Sept 

7176 

Cumu] at ive 
Error/Hr 


0011 


0010 


Avur.ige 

Time 

Botwf-cn 

Errors 

(MTBF^O 


909 


1008 


km 


Table  7.  Channel  Management  Module  -  Reliability  Data  Summary 


Monthly 

Monthly 

Cumulative 

Cumulative 

Cumulat  i' 

Test  Hrs 

Errors 

Test  Hrs. 

Errors 

Error/Hr 

Month 

(T) 

(F) 

(Tj) 

(Fj) 

1979  Feb 

140 

1 

140 

1 

.00714 

Mar 

140 

280 

Apr 

140 

- - - 

420 

-  -  - 

May 

140 

1 

560 

2 

.  0036 

June 

140 

i 

700 

•July 

188 

' 

888 

1 

Aug 

188 

1076 

Sept 

188 

1 

126A 

i 

3 

.0024 

Oct 

305 

1569 

.  _  -  .  - -  -  - - 

.... - 

Nov 

390 

1959 

1979  Dec 

489 

1 

2448 

4 

.0016 

1980  Jan 

528 

1 

2976 

5 

.0017 

Fob 

480 

2 

3456 

7 

.002 

Mar 

431 

2 

3887 

9 

.0023 

Apr 

624 

2 

4511 

11 

.0024 

May 

418 

3 

4929 

14 

.0028 

June 

457 

5386 

July 

Aug 

1790 

Sept 

7176 

Average 

Time 

Between 

Errors 

(MTBFj,) 

140 


2  78 


417 


625 


588 

500 

435 

417 

357 
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Aiiniilativo  Krror  Rate  (Error/Hr) 


Table  8.  Network  Management  -  Reliability  Data  Summary 


Month 


1979  Feb 
Mar 

Apr 

May 

June 

July 

Aug 

Sept 

Oct 
Nov 
1979  Dec 


1980  Jan 
Feb 
Mar 

Apr 

May 

June 

July 

Aug 

Sept 


Monthly 
Test  Hrs 
(T) 


305 

390 

489 


528 

480 

431 

624 

418 

457 


Monthly 

Errors 

(F) 


1790 


3 

0 


2 

4 

3 

3 

1 

0 


Cumulative 
Test  Hrs. 

(Tj.) 


305 

695 

1184 


1712 

2192 

2623 


3247 

3665 

4122 


5912 


Cumulative 

Errors 

(Fj,) 


11 

15 

18 


21 

22 


Cumulat ive 
Error/Hr 

(A,) 


.0098 

.018 

,0064 

.0068 

.0069 

.0065 

.0060 


Average 

Time 

Between 

Errors 

(MTBFj,) 


102 

56 

156 

147 

145 

154 

167 
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NM  cumulative  error  rate  was  measured  to  be  Xj;  =  22/4122  =  .00534 
error/hr.  or  187  hours  MTBF,  NM  software  has  been  characterized  as 
moderately  complex;  it  has  the  highest  predicted  error  rate  of  the 
DABS  software  modules  which  were  studied.  Although  its  error  rate 
appears  to  be  decreasing  (see  Figure  9),  there  are  not  sufficient  cur 
rent  data  to  support  trend  analysis.  Data  reported  during  the  pre¬ 
diction  test  interval  also  indicate  a  decreasing  error  rate  trend. 
During  1790  hours  of  test  there  were  no  reported  errors.  However, 
a  large  portion  of  the  apparent  reliability  improvement  may  be  due 
to  a  lessening  of  the  severity  of  the  test  environment.  The  formal 
NM  test  which  was  run  to  demonstrate  compliance  with  operational 
requirements  liad  been  completed  in  June  1980.  Consequently  the  NM 
software  may  have  been  operating  in  reduced  data  and  requirements 
environments  during  the  July  to  September  time  frame. 

U.J  RELIABILITY  KV.ALUATIQN  OF  M'm  MODULES. 

The  Moving  Target  Detector  (MID)  and  Radar  Data  Acquisition  Subsystem 
(RDAS)  programs  are  integral  to  the  sensor  track  software  which  is 
required  to  acquire  and  track  DABS  and  ATCRBS  aircraft. 

As  noted  in  Table  9,  the  MTD  and  RDAS  module  was  tested  for  only  3 
months  resulting  in  1499  test  hours  and  3  chargeable  errors  for  an 
error  rate  of  .002  error/hr.  or  500  hours  MTBF.  Tlie  data  in  Figure 
10  show  a  constant  error  rate,  with  a  slight  increasing  trend  which 
is  influenced  by  the  paucity  of  data.  During  the  prediction  test 
interval  no  errors  were  reported  against  the  IflD  and  RDAS  module. 


RELIABILITY  EVALUATION  OF  SYSTEM  SOFTWARE  MODULE. 

System  (SYS)  software  refers  to  all  the  software  required  to  calibrat 
and  initialize  DABS  and  recover  from  hardware  failures.  SYS  also  ct)n- 
tains  standardized  software  support  utilities. 
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Table  10.  System  Software  Module  -  Reliability  Data  Summary 


Month 

Monthly 
Test  Hrs 
(T) 

Monthly 

Errors 

(F) 

Cumulative 
Test  Hrs. 

(Tj) 

Cumulative 

Errors 

Cumulative 

Error/Hr 

(A,) 

Average 

Time 

Between 

Errors 

(MTBFj.) 

1979  Feb 

Mar 

Apr 

1 

May 

140 

4 

140 

4 

.029 

34 

June 

140 

1 

280 

5 

.018 

56 

July 

188 

3 

468 

8 

.017 

59 

Aug 

188 

2 

656 

10 

.015 

67 

Sept 

]88 

1 

4 

844 

14 

1 

.017 

1 

59 

Oct 

305 

1 

1149 

15 

.013 

77 

Nov 

390 

1 

1539 

16 

.010 

100 

1979  Dec 

489 

0 

2028 

1980  Jan 

528 

0 

2556 

Feb 

480 

1 

3036 

17 

.0056 
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Mar 

431 

0 

3467 

Apr 

624 

4091 

May 

418 

1 

4509 

18 

.004 

2  50 

June 

457 

0 

4966 

July 

prediction  that  one  error  will  occur  during  the  prediction  test  inter¬ 
val.  Three  chargeable  errors  were  reported,  a  number  which  is  quite 
high.  The  occurrence  of  three  or  more  errors  when  only  one  is  expected 
should  occur  no  more  than  8  percent  of  the  time.  Therefore,  the 
prediction  is  marginally  acceptable. 

The  instantaneous  error  rate  of  .000386  error/hr.  at  T  =  6756  hours 
was  predicted  using  the  model  X  =  .78T~'®^^.  Based  on  the  above 
comparison  between  predicted  and  actual  numbers  of  failures,  the 
instantaneous  failure  rate  is  expected  to  be  somewhat  optimistic. 

A. 9  RELIABILITY  EVALUATION  OF  TOE  SURVEILLANCE  PROCESSING  MODULE . 

The  Surveillance  (SURV)  processing  module  is  responsible  for  tracking 
targets,  correlating  radar  reports  with  beacon  reports  or  tracks  and 
for  maintaining  the  surveillance  file. 

Tlie  SURV  module  accrued  5386  hours  of  test  during  which  41  chargeable 
errors  were  reported.  Its  cumulative  error  rate  at  T  =  5386  was  .0076 
error/hr.  or  131  hours  MTBF.  The  data  used  in  the  reliability  anal¬ 
ysis  is  contained  in  Table  11.  Figure  12  displays  the  reliability 
growth  model.  It  should  be  noted  that  the  SURV  module  exhibited 
decreasing  error  rates,  but  at  different  rates.  The  change  in  slope  of 
the  curve  may  be  attributed  to  variations  in  the  test  environment,  to 
delays  in  documenting  the  errors  or  to  delays  in  implementing  correc¬ 
tive  action.  All  three  situations  have  been  identified  as  causative 
factors  which  perturb  reliability  growth  models.  Note  that  the 
growth  curve  was  generated  using  weighted  least  squares  which  stresses 
current  data.  The  slope  of  the  line  favors  the  current  trend  rather 
than  the  overall  trend. 

Based  on  the  cumulative  growth  model,  X^  =  .  1067T  the  predicted 

A^;  at  T  =  7176  hours  is  .006983  error/hr.  which  is  equivalent  to  a 
total  of  50.1  expected  errors.  This  implies  a  prediction  of  9.1 
errors  during  the  1790-hour  prediction  test  interval.  Seven  (7) 
errors  were  reported  against  the  SURV  module,  a  number  which  compares 
favorably  with  the  prediction. 

Tlie  instantaneous  growth  model,  Xj.  =  .0739T  predicts  an  error 

rate  of  .00484  error/hr.  or  207  hours  MTBF  at  T  =  7176  test  hours. 


5.  INTEGRATED  DABS  HARDWARE  AND  SOFTWARE  RELIABILITY  MODEL. 


5.1  SOFTWARE  SUBSYSTEM  RELIABILITY  MODEL. 

The  reliability  block  diagram  of  the  DABS  software  is 


( 

Performance 

Message  Routing 

Monl torlng 

&  Data  Link 

System 

i 

'Surveillance 

! 

Data 

Software 

Processing 

Extraction 

— 

- - - 

_  .  . 

— 

Channel 

Network 

L 

J 

MTD  & 

i 

L_ 

■Management 

Management 

r  , 

RDAS 

The  reliability 


model  which  corresponds  to  the  above  block  diagram  is 


^.w.  =  TT’^i 

i=l 


where  Rg  ^  =  Reliability  of  the  software  subsystem 

R^  =  Reliability  of  each  module,  i=l,  9 

It  was  noted  in  Section  A  of  this  report  that  the  reliability  growth 
model  used  to  measure  DABS  software  reverts  to  a  constant  failure  rate 
model  at  the  conclusion  of  the  test/analyze/fix  process  underway  during 
a  test  program.  Therefore,  the  reliability  model  for  operational 
software  is  identical  to  the  reliability  model  used  by  the  FAA  to 
model  hardware  reliability.  It  is  characterized  by  an  exponential 
distribution  of  times  between  errors  or  it  may  be  stated  as  a  Poisson 
distribution  of  the  number  of  errors  within  a  specified  time  interval. 

Using  terminology  similar  to  that  used  by  the  FAA  in  the  hardware 
reliability  model, 


S.W. 


N 
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where  W  ~  error  rate  of  the  total  software  program 

X.  =  error  rate  of  a  software  module  for  i=l,  9. 

As  noted  earlier,  the  error  rates  of  the  modules  are  changing  because 
of  the  results  of  debugging  the  software.  Therefore,  the  reliability 
prediction  will  be  made  in  terms  of  accumulated  software  test  time. 

A  summary  of  reliability  predictions  for  the  DABS  software  modules  is 
presented  in  Table  12.  In  summary,  it  states  that  a  chargeable  error 
will  occur  within  the  DABS  software  every  53  test  hours.  For  compari¬ 
son  purposes.  Table  13  contains  a  summary  of  module  critical  error 
rates . 

It  was  noted  earlier  that  the  error  rate  of  a  module  may  improve 
dramatically  once  the  module  is  removed  from  a  test  environment.  An 
improvement  factor  of  5  was  noted  by  the  author  in  a  similar  study. 

It  is  not  implied  that  the  same  factor  is  applicable  to  DABS  software, 
but  if  it  were,  the  time  between  chargeable  errors  would  increase  to 
only  265  hours. 

The  software  reliability  model  makes  no  provision  for  software  repair 
in  the  event  of  failure.  The  DABS  system  is  structured  to  provide 
reconfiguration  in  the  event  of  certain  hardware  failures.  Critical 
software  failures  will  generally  fail  the  system.  Also,  redundancy 
features  of  hardware  do  not  apply  to  software.  If  two  redundant  pro¬ 
cessors  encounter  the  same  logical  software  error,  and  if  the  error  is 
critical,  both  processors  and  therefore  the  computer  will  fail. 


5.2  DABS  INTEGRATED  SYSTEM  (HARDWARE  AND  SOFTtv'ARi:)  RELIABILITY  MODEL. 

Hie  reliability  block  diagram  which  combines  hardware  with  software 
elements  of  DABS  is 


Hardware  i 
Elements _ j 


Software  I 
Elements  I 


The  reliability  model  is  x 

Translated  into  the  effective  failure  rate  (X^pp)  model  used  by  FAA, 

^  DABS  ~  Xj^Pp(Hardware)  +  X(Software).  Based  on  data  contained  in'  Re¬ 
port  No.  FAA-RD-80-36 ,  "Discrete  Address  Beacon  System  (DBAS)  Base¬ 
line  Test  and  Evaluation",  by  M.  Holtz,  Xppp  =  .001736  failure/hr. 
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Table  12.  Summary  of  Software  Reliability  Predictions 

Reliability  Predictions 


Software  Module 

Number  of  Errors 
Within  Prediction 
Interval 

Number  of 
Errors/ 
Test  Hour 

Average  Time 
Between 
Errors 

Communications 

2.9 

.001020 

976 

Performance  Monitor 

3.5 

.002030 

494 

Message  Routing  & 

Data  Link 

0.9 

.000417 

2400 

System  Software 

1 

.000386 

2588 

Surveillance  Processing 

9.1 

.004840 

207 

Data  Extraction 

No 

Predi ction 

.00037 

2703 

Channel  Management 

No 

Prediction 

.00260 

385 

Network  Management 

No 

Prediction 

.00534 

187 

MTD  &  RDAS 

No 

Predi ction 

.00200 

500 

TOTAL 

.019 

53 

42 


Table  13.  Summary  of  Module  Critical  Error  Rates 


Software  Module 

Test 

Hours 

Number  of 
Critical 
Errors 

Critical  Error 
Rate  -  Critical 
Error/Hr. 

COM 

4966 

5 

.001 

PM 

4966 

5 

.001 

MR  &  DL 

4966 

0 

— 

DEX 

5386 

1 

.00019 

CM 

5386 

4 

.00074 

NM 

4122 

2 

.000485 

MTD  &  RDAS 

1499 

1 

.000667 

SYS 

4966 

7 

.0014 

SURV 

5386 

6 

.00111 

.00662 


Note:  During  July,  August  and  September  of  1980,  4  critical  errors 

were  reported  during  1790  hours  of  test.  The  error  rate  of 
.00223  error/hr  is  equivalent  to  MTBCF  of  448  hours. 
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.001736  +  .019  = 


Using  all  chargeable  software  errors, 

.020736  failure/hr.  or  48  hours  MTBF.  Using  only  critical  software 

errors,  X„,„„  =  .001736  +  .00223  =  .003966  fail/hr.  or  252  hours  MTBF. 
UAjdd 
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APPENDIX  A 


Review  and  Critique  of  the  Available 
Hardware  Reliability  Model  and  the  Hardware 
Reliability  Prediction  for  the  DABS 


FAA  Report  No.  FAA-NA-78-31,  "Plan  for  the  Reliability  and  Main¬ 
tainability  Evaluation  of  the  Discrete  Address  Beacon  (DABS)  Engineer¬ 
ing  Laboratory  Models,"  contains  the  hardware  reliability  model  and 
reliability  prediction  for  the  DABS.  The  report  also  addresses  the 
failure  reporting,  data  collection,  data  processing  system  and  the 
criteria  which  will  be  used  to  evaluate  (measure)  hardware  reliability 
of  engineering  laboratory  models. 

The  critique  presented  herein  addresses  only  those  portions  of 
the  report  which  deal  with  the  DABS  hardware  reliability  models  and 
the  reliability  prediction.  Review  and  critique  of  the  failure  data 
collection,  processing  and  analysis  procedures  are  outside  the  scope 
of  this  task. 

The  FAA  report  describes  the  construction  and  use  of  a  series  of 
Einhorn  equations  (models)  which  transform  mean-time-between-failure 
(MTBF)  and  mean-t ime-to-repair  (MTTR)  of  an  equipment  into  effective 
failure  rate  for  the  equipment.  In  addition,  a  method  is  pro¬ 

vided  by  which  effective  failure  rates  of  2  or  more  equipments  may  be 
combined  to  produce  a  subsystem  effective  failure  rate.  The  models 
of  the  DABS  subsystems  are  well  prepared  and  documented.  The  comments 
which  follow  address  minor  points  of  the  modeling  process  and  several 
major  topics  which  were  not  addressed  in  the  report,  but  which  might 
be  candidates  for  inclusion  in  a  revision  to  the  document. 

LFNFRAL  COMMENTS : 

1.  The  predicted  mean-t ime-between-failures  (MTBF)  of  a  single 
channel  sensor  is  774  hours,  a  value  considerably  lower  than  the  1,000 
hours  specified  in  the  engineering  requirement.  There  is  nothing  in 
the  document  to  indicate  that  appropriate  improvements  such  as 
redesign  or  use  of  high  reliability  parts  will  be  employed  to  improve 
system  reliability.  As  stated  in  Report  No.  FAA-RD-80-36,  DABS 
measured  MTBF  is  575  hours  but  is  increasing.  The  FAA  should  monitor 
test  results  closely,  continue  to  measure  system  MTBF  during  develop¬ 
ment  testing  and  implement  effective  corrective  actions  to  improve 
system  MTBF  to  meet  the  specification. 

2.  The  state  diagram  technique  used  to  model  DABS  hardware  reliability 

appears  to  generate  which  is  pessimistic.  Significant  terms  in 

the  calculation  of  obtained  by  multiplying  the  failure  rate 

of  a  specific  hardware  configuration  by  the  probability  of  failure  in 
the  configuration  for  the  entire  anticipated  mission.  However,  the 
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roost  likely  time  of  failure  for  equipments  having  MTBF  >>  mission  time 
is  near  the  midpoint  of  the  mission.  Hence,  the  calculated  probabilities 
of  failure  are  nearly  doubled. 

3.  The  report  should  contain  a  brief  but  complete  description  of  the 
DABS  mission.  A  complete  reliability  evaluation  plan  should  describe 
the  anticipated  mission  or  a  standardized  mission  which  will  be  used 
for  reliability  measurement.  Mission  identification  should  identify 
and  describe  all  mission  phases,  their  duration  and  anticipated 
environments.  The  results  of  the  mission  analysis  should  then  be  merged 
with  the  results  of  a  systems  analysis  which  then  identifies  the  full 
complement  of  equipment,  including  reliability  block  diagrams,  which 
will  be  used  to  measure  reliability  during  each  mission  phase.  Also 
Included  are  alternate  modes  of  operation  and  success/failure  criteria, 

4.  The  reliability  equations  are  very  general  and  optimistic  because 
they  include  the  probability  of  repairing  equipments  without  consider¬ 
ing  the  number  of  repairmen,  number  of  spares  or  administrative  delays 
which  may  prolong  maintenance  time.  The  FAA  equations  are  applicable 
only  if  an  infinite  number  of  spares  and  repairmen  are  instantly 
available  at  each  operational  site.  If  reasonable  constraints  were 
placed  on  the  above  model  parameters,  predicted  reliability  would 
decrease. 

5.  The  FAA  report  specifically  states  that  "special  reliability  tests" 
will  not  be  conducted  and  that  objectives  of  the  reliability  and 
maintainability  (R&M)  evaluation  can  be  achieved  using  FAA  performance 
tests.  It  should  be  recognized  that  not  all  performance  tests  will  be 
applicable  to  the  measurement  of  DABS  R&M.  The  report  should  contain 
the  results  of  an  analysis  of  the  anticipated  test  program  which  would 
describe  the  quantity  and  quality  of  the  anticipated  data  and  why  the 
data  can  be  used  for  R&M  measurement. 

6.  The  "estimation"  of  equipment  MTBF  should  include  the  calculation 
of  confidence  intervals  for  equipment  and  system  MTBF;  i.e.,  an 
interval  which  contains  the  true  but  unknown  MTBF  with  stated 
probability. 


S P EC I  FI C  COMMENTS : 

1.  Rage  28  Second  Paragraph 

Per  this  paragraph.  A  statistical  test  will  be  performed  to 
determine  if  the  exponential  distribution  is  appropriate  to  describe 
time  to  failure  and  time  to  repair  data.  The  report  should  describe 
alternative  statistical  techniques  for  data  analysis  if  the  exponential 
distribution  fails  to  adequately  describe  these  time  data.  This  is 
especially  Important  for  time  to  repair  which  is  often  modeled  using 
the  log-normal  distribution. 


2.  Pages  39/40 


The  reliability  block  diagram  for  two  redundant  equipments  with 
a  switch  is 


The  reliability  model  for  the  above  system  is 
^  ®  ^SWITCH  ® 

where  X  =  failure  rate  of  176  K  memory  string 

t  =  mission  duration 

R  =  reliability  of  switching  element 

o  W  X  i  U  il 

3.  Titles  of  Figures  1,  2,  4  and  12 

These  figures  are  titled  "reliability  models"  but  a  more 
appropriate  title  is  "reliability  block  diagram."  The  reliability 
model  is  usually  defined  as  the  equation  which  transforms  MTBF  into 
probability  of  success. 


APPENDIX  B 


A  Recommended  Software  Reliability  Failure 
Reporting  System  for  DABS 


The  FAA  currently  uses  the  DABS  Trouble  Report/Change  Proposal 
(Figure  B-1)  to  document  software  errors.  Additional  analysis  and 
follow-up  are  documented  on  DABS  Trouble  Report/Change  Proposal  Update 
Worksheet  (Figure  B-2) .  While  the  reporting  system  was  not  structured 
specifically  to  provide  data  suited  for  reliability  analysis,  the 
forms  do  provide  most  of  the  required  information  when  completed  in 
accordance  with  the  Trouble  Report  Users  Guide. 

The  difficulties  encountered  when  using  DABS  Trouble  Reports  (TR) 
were  primarily  lack  of  completeness  and  lack  of  error  classification. 
These  weaknesses  in  the  present  system  can  be  corrected  by  instituting 
an  editorial  review  of  the  software  errors  as  TRs  are  initiated,  com¬ 
pleted  and  classified.  A  glaring  weakness  in  the  procedure  can  be  cor¬ 
rected  by  ensuring  that  the  TRs  contain  the  date  of  occurrence  of  the 
error  as  well  as  the  date  of  TR  initiation.  It  is  recommended  that  a 
representative  of  the  reliability  engineering  group  participate  in  the 
editorial  review  because  much  of  the  data  are  needed  for  reliability 
analysis  purposes'. 

As  soon  as  error  follow-up  identifies  causes  for  the  initiation 


of  a  TR  it  should  be  classified  with  regard  to  CATEGORY  and  SEVERITY. 
With  regards  to  CATEGORY,  the  following  definitions  are  recommended 
for  use  by  FAA. 

Error  Source 
Code 

Error  Source 

Description 

0 

Requi renents 

Source  of  problem  is  changing, 
ill  conceived  or  poorly  stated 
performance  requirement. 

1 

Design 

Source  of  problem  is  in  prelim¬ 
inary  or  detailed  design. 

2. 

Coding 

Source  of  problem  is  an  error  in 
implementing  the  design  or  code. 

3. 

Maintenance 

Source  of  problem  is  an  error  in¬ 
troduced  in  process  of  trying  to 
fix  a  previous  error. 

A. 

Not  Known 

Source  of  error  not  known. 
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Figure  B-1.  O.Mir,  Trouble  Report/CT.ingo  Proposal 
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dabs  trouble  RCI'ORT/CHANCE  I'ROt'OSAL  UPDATE  WORKSHEET 

f  Nl  i  R  AT  L  (  AS1  ONI  OF  T  HE  f  OL  I  NC  TO  iOl  NT  IF  V  Rf  PORT 

Rri'ORT  chance 

I  0(|H  NO.  _  _  PHO<‘OSAL  NO 

SHORT  ni  SCRifM  (ON 

SOI  UllOH'COM*i|  HTARY 


TROUBLE 
HI  l‘OH  T  NO. 


MOOUl  fS  CHANCtO  ISR  hOOUIE  name.  MtA  PART  KUMBfcftI 


INSTAI  lAKONS  AfffclEO 


OT  Mt  R  FORMS  iNiTiA  T  to 

€CN  OCN 


change  test  results 

r  ]  Sw  C  HANCF  PROPOSE  0  DOfS  NOT  C  ORR|  C  T  PROBl EH 

(]  S**  CHANCE  TESTED  AND  Bl  AQT  FOR  SYSTEM  TEST  (SC  MR  ATTACHED) 

(^J  HA  correction  rOES  NOT  SOIVE  PROBIEM 

[}  HA  fix  TESTFO  and  ready  for  iMPIE»'EN  T  AT  ION  «EcN  ATTACHED) 

IROUBLE  CATE  CORY 

T]  MHARnAARE  PROBLEM 
(]  S  -SOF  TAARE  PROBLEM 
(J  O-DOCUMENTATION  PROBL  EM 
f  I  C  'DE  SiC,N  PROBE  EM 
C1  R"NE»  REQUIREMENT 

NA  FOR**  A3tS3ET  7*) 


Figure  B-2.  DABS  Trouble  Report/Change  Proposal 
Update  Worksheet 


Errors  should  also  be  classified  as  to  type: 


Error  Type  Code 


Type  of  Software  Errors 


A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

X 


Computational 
Logic  Errors 
Data  Input  Errors 
Data  Handling  Errors 
Data  Output  Errors 
Interface  Errors 
Data  Definition  Errors 
Data  Base  Errors 
Operational  Errors 
Other 

Documentation  Errors 
Trouble  Report  Rejection 


Trouble  reports  should  be  classified  as  with  regard  to  SEVERITY  in 
accord  with  the  following  definitions: 

Chargeable  Errors: 

1.  Critical  -  An  error  in  the  softi<;are  which  causes  a 
significant  or  total  loss  of  operational  system  capability. 

2.  Non-Critical  (Major)  -  An  error  in  the  software  which 
causes  an  erroneous  response  in  the  operational  system.  An  error 
in  this  classification  may  not  be  recognized  as  such  by  a  trained 
observer  due  to  the  self  repair  inherent  in  the  system. 

Mon-Chargeable  Errors: 

3.  Other  (Minor)  -  An  error  which  has  no  measurable  ef¬ 
fect  on  the  operational  system  or  are  of  unkown  cause  at  this  time 
(hardware/software/cockpit).  Errors  of  unknown  cause  would  be  charged 
against  the  DABS  system  rather  than  the  software. 

A.  No  Count  -  A  trouble  report  which  was  erroneously  attributed 
to  software  errors.  In  addition,  change  proposals,  enhancements, 
duplicate  trouble  reports  and  "cockpit  errors"  are  included. 
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AI'PKNDIX  C 


LISTING  OF  DABS  SOFTWARE  TROUBLE  REPORTS 
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!.0152  DS-C-0120  0067  lOIiAD  DCCTIIIATION  M.'C.CAGKS  DAB007  PM  H«12/l  79  11 

COlbO  DC  C  0121-0048  lllnPLEnCNT  CHANCE  ATARS  DAD007  A3ARS  *12/1'  791110 

'.0158  DC  S  0120  0048  24IIIE0AL  SENCOR  STATUS  DAD007  NM  ««12/i'  79  11 

C0154  -  SNS  CALL  REPLIES  LOCKOUr  DAC007  NM  •l»12/;-  79  I 

C0156  DS-S  0122-0048  23rHK  COORDINATION  Mfc CCAOE  DAD007  NM  •12/1-  79M10 

C0157  -  SNS  FADE  OF  MWPAH  DAD007  SURV  <l*12/l-  79  1 

S0159  DS-S  0140-0003  00  CONDITIONS  REMOTE  DATA  DAB007  NM  12/1-  79M10 

',.0161  DS-S-0124'0G48  25NM  2-1  TKANSIllON  DAB007  NM  t»»12/l-  79  11 
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S0149  DS  E-0131-0067  11  INCORRECT  ACTh'BS  REPLY  A  EL  DAB007  CURV  Ilitl270-  79  1  1 

b0179  DS -S-01 37  0067  1  ONU.MBER  OF  CONN  SCN-..ORS  =  0  DAB007  PM  ••01/.:  GO  11 

CG169  DS-C-0135  0067  MALL  CALL- TO-CEiAST  DATA  REQU  DA3007  NM  «*C17C'  SO  II 

S0175  DS  G-0136-0046  31DATA  STOP  WITH  ZERO  ID  CAB007  NM  ••01/1.  SO  11 

COieO  DS-S -01 40-0049  33REQ  FOR  PRIM  IN  CLnT  CFLLDA2007  NM  *01/1  SOS  5S 

P0174  DS-S-0139-0048  32  NET  MGT  HI  COM  lEST  DAB007  NM  ••01/0;  80  3 

S0167  DS  S-0133-0067  13  MULTISUE  ADAPTATION  DAE007  SA  B»01/C'  SC  11 

G0165  DS  S-0141  0048  34  OVERLOAD  COMM  J<  CAAT  FACIL  DAB007  NM  BBID/E;  79  11 

C0166  DS-S-0134  0048  30  ChFCKS  FOR  UNLK  LOCKED  FILFDAS007  NM  ••12/2-  79  11 

r-0164  DE-S-01C2-0067  12  LOST  E-ABS  DUE  TO  DAAT  DAB007  NM  ••12/S.  79  11 

N0004  DS'S-0I30-007B  00  INCOfi  ROLL-CALL  REPLIES  DAB006  CM  BBIB/Bl  79  11 

50188  DS-S-01 54-0009  00  ATCRBS  TRACK  HANG  DAD007  SURV  0l/2:  BOW  7 

50189  DS-S-0165  0019  00  FR(INT,BaCK  RADAR  FAN.GE  MACKDAB007  CM  •Ol/D:  SO  5C 

S0255  DC  LOSS  OF  DATA  ON  MULTI  DISS  DAB007  CO/TM  01/2'  rOG  OS 

N.'.'OOS  DS-N-0CC4-0036  00  INC  BIT  IN  SURvETlL  FILE  DAS007  SURV  01/1!  'BOW  7 

00193  DS-S -0146- 0003  00  fPROR  IN  DOWNLINK  ELM  FBOC  CAB007  CM  02/0.  BOW  7 

50194  DS- S-0M5- 0002  00  LOSING  ALL-CA(L  SYNC  DAB007  CM  02/0.  80K10 

50195  DS-S-0144  0001  00  BAD  BIAS  REGISTER  DAB007  SYS  02/0.  SOW  55 

S01B7  DS  S-OM2-0079  00  UPDATED  CAL  CURVES  DAE007  5A  01/21  BOMIO 

N0006  DS- N- 0001 -0C4S  35  SUFTWR  TESTS  TOR  MBL  ?  MBH  DAB007  SURV  Ol/lf  BOMIO 

S0202  DS-A-OOOl -0001  00  IMPLEMENT  INTERIM  ATARS  DABOOB  AG-ARS  »02/0'  BOMIO 

N0007  DS-S-0169-0023  00  RLOUESTED  SOF TWR  CHNGS  DAB007  MTD  02/0'  BCE  5S 

S0208  DS  -S-0150-0005  00  I  OCAL  SENSOR  SbCuNDARY  DAB007  NM  02/1':  BOW  7 

50207  DS  S- 0149-0003  00  CYblLM  SPECIAL  M.  .DE  FLAG  DAD007  NM  02/1!  BOW  7 

N2001  BK  restarting  ARIES  DAS006  SURV  02/21  SOB  OS 

50217  J5  LOSS  OF  ATCRBS  TARGETS  DA9007  SURV  02/0!  BOW  OS 

S0210  SS  ATCRBS  REM  TRK  DATA  PASSED  DAB007  NM  02/2:  BOS  OS 

50213  DS  S-OI 52-0006  OOINCORR  OISSEM  OF  Cl  DIN  MSGS  DABOO?  COMM  02/2'  SOG  5S 

50212  DS-S-0151-0005  OOHI  PR  I  COMM  BUFF  BACK-UP  DAB007  COMM  02/2'  BOG  5S 

50214  DS-S-01 53-0007  OOCIDIN  FAIL  TO  CHK  ZERO  VALUEDAB007  COMM  02/2'  BOG  5S 

50190  DS-S-01 54-0008  00  CHANGING  EXT  DATA  STREAMS  DAB007  NM  ••01/22  80  3 

NOOOB  DS -N-0003-0008  00  CHANNEL  MGT  INTERPDG  SPAC  DAB007  8A  *01/2'  SOMIO 

TOOOl  DS-U-0001-0084  00  TRACK  ALERT  MESSAGE  DAB006  NM  *02/1=  8CS  4S 

S0244  DS-S-01 57-0037  OOINCOR  LINK  SWITCH  CONTR  DAB007  SA  03/0-  BOG  5S 

50218  DS-S-01 56-0010  00  COMPILE  ERROR  IN  DABOOB  RELDAB007  SYS  *02/2=  SOM  5S 
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